Safety signal identification for COVID-19 bivalent booster vaccination using tree-based scan statistics in the Vaccine Safety Datalink.

BACKGROUND: Traditional active vaccine safety monitoring involves pre-specifying health outcomes and biologically plausible outcome-specific time windows of concern, limiting the adverse events that can be evaluated. In this study, we used tree-based scan statistics to look broadly for >60,000 possible adverse events after bivalent COVID-19 vaccination. METHODS: Vaccine Safety Datalink enrollees aged ≥5 years receiving Moderna or Pfizer-BioNTech bivalent COVID-19 vaccine through November 2022 were followed for 56 days post-vaccination. Incident diagnoses in inpatient or emergency department settings were analyzed for clustering within the hierarchical ICD-10-CM diagnosis code "tree" and temporally within post-vaccination follow-up. The conditional self-controlled tree-temporal scan statistic was used, conditioning on total number of cases of each diagnosis and total number of cases of any diagnosis occurring during the scanning risk window across the entire tree. P = 0.01 was the pre-specified cut-off for statistical significance. RESULTS: Analysis included 352,509 doses of Moderna and 979,189 doses of Pfizer-BioNTech bivalent vaccines. After Moderna vaccination, no statistically significant clusters were found. After Pfizer-BioNTech, there were clusters of unspecified adverse events (Days 1-3, p = 0.0001-0.0007), influenza (Days 35-56, p = 0.0001), cough (Days 44-55, p = 0.0002), and COVID-19 (Days 52-56, p = 0.0004). CONCLUSIONS: For Pfizer-BioNTech only, we detected clusters of: (1) unspecified adverse effects, as have been observed in other vaccine studies using this method, and (2) respiratory disease toward the end of follow-up. The respiratory clusters were likely due to overlap of follow-up with the spread of respiratory syncytial virus, influenza, and COVID-19, i.e., confounding by seasonality. The untargeted nature of the method and its inherent adjustment for the many diagnoses and risk intervals evaluated are unique advantages. Limitations include susceptibility to time-varying confounding, lower statistical power for assessing risks of specific outcomes than in traditional studies targeting fewer outcomes, and the possibility of missing adverse events not strongly clustered in time or within the "tree."

Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation.

Congress mandated the creation of a postmarket Active Risk Identification and Analysis (ARIA) system containing data on 100 million individuals for monitoring risks associated with drug and biologic products using data from disparate sources to complement the US Food and Drug Administration's (FDA's) existing postmarket capabilities. We report on the first 6 years of ARIA utilization in the Sentinel System (2016-2021). The FDA has used the ARIA system to evaluate 133 safety concerns; 54 of these evaluations have closed with regulatory determinations, whereas the rest remain in progress. If the ARIA system and the FDA's Adverse Event Reporting System are deemed insufficient to address a safety concern, then the FDA may issue a postmarket requirement to a product's manufacturer. One hundred ninety-seven ARIA insufficiency determinations have been made. The most common situation for which ARIA was found to be insufficient is the evaluation of adverse pregnancy and fetal outcomes following in utero drug exposure, followed by neoplasms and death. ARIA was most likely to be sufficient for thromboembolic events, which have high positive predictive value in claims data alone and do not require supplemental clinical data. The lessons learned from this experience illustrate the continued challenges using administrative claims data, especially to define novel clinical outcomes. This analysis can help to identify where more granular clinical data are needed to fill gaps to improve the use of real-world data for drug safety analyses and provide insights into what is needed to efficiently generate high-quality real-world evidence for efficacy.

Use of Electronic Health Record Data for Drug Safety Signal Identification: A Scoping Review.

INTRODUCTION: Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance. METHODS: To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices. RESULTS: We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations. CONCLUSION: Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.

Valsartan, Losartan and Irbesartan use in the USA, UK, Canada and Denmark after the nitrosamine recalls: a descriptive cohort study.

OBJECTIVES: To examine valsartan, losartan and irbesartan usage and switching patterns in the USA, UK, Canada and Denmark before and after July 2018, when the first Angiotensin-Receptor-Blocker (ARB) (valsartan) was recalled. DESIGN: Retrospective cohort study. SETTING: USA, Canadian administrative healthcare data, Danish National Prescription Registry and UK primary care electronic health records. PARTICIPANTS: Patients aged 18 years and older between January 2014 and December 2020. INTERVENTION: Valsartan, losartan and irbesartan. MAIN OUTCOME: Monthly percentages of individual ARB episodes, new users and switches to another ARB, ACE inhibitors (ACEI) or calcium channel blockers containing products. RESULTS: We identified 10.8, 3.2, 1.8 and 1.2 million ARB users in the USA, UK, Canada and Denmark, respectively. Overall proportions of valsartan, losartan and irbesartan use were 18.4%, 67.9% and 5.2% in the USA; 3.1%, 48.3% and 10.2% in the UK, 16.3%, 11.4% and 18.3% in Canada, 1%, 93.5% and 0.6% in Denmark. In July 2018, we observed an immediate steep decline in the proportion of valsartan use in the USA and Canada. A similar trend was observed in Denmark; however, the decline was only minimal. We observed no change in trends of ARB use in the UK. Accompanying the valsartan decline was an increase in switching to other ARBs in the USA, Canada and Denmark. There was a small increase in switching to ACEI relative to the valsartan-to-other-ARBs switch. We also observed increased switching from other affected ARBs, losartan and irbesartan, to other ARBs throughout 2019, in the USA and Canada, although the usage trends in the USA remained unchanged. CONCLUSION: The first recall notice for valsartan resulted in substantial decline in usage due to increased switching to other ARBs. Subsequent notices for losartan and irbesartan were also associated with increased switching around the time of the recall, however, overall usage trends remained unchanged.

A broad assessment of covid-19 vaccine safety using tree-based data-mining in the vaccine safety datalink.

BACKGROUND: Except for spontaneous reporting systems, vaccine safety monitoring generally involves pre-specifying health outcomes and post-vaccination risk windows of concern. Instead, we used tree-based data-mining to look more broadly for possible adverse events after Pfizer-BioNTech, Moderna, and Janssen COVID-19 vaccination. METHODS: Vaccine Safety Datalink enrollees receiving ≥1 dose of COVID-19 vaccine in 2020-2021 were followed for 70 days after Pfizer-BioNTech or Moderna and 56 days after Janssen vaccination. Incident diagnoses in inpatient or emergency department settings were analyzed for clustering within both the hierarchical ICD-10-CM code structure and the post-vaccination follow-up period. We used the self-controlled tree-temporal scan statistic and TreeScan software. Monte Carlo simulation was used to estimate p-values; p = 0.01 was the pre-specified cut-off for statistical significance of a cluster. RESULTS: There were 4.1, 2.6, and 0.4 million Pfizer-BioNTech, Moderna, and Janssen vaccinees, respectively. Clusters after Pfizer-BioNTech vaccination included: (1) unspecified adverse effects, (2) common vaccine reactions, such as fever, myalgia, and headache, (3) myocarditis/pericarditis, and (4) less specific cardiac or respiratory symptoms, all with the strongest clusters generally after Dose 2; and (5) COVID-19/viral pneumonia/sepsis/respiratory failure in the first 3 weeks after Dose 1. Moderna results were similar but without a significant myocarditis/pericarditis cluster. Further investigation suggested the fifth signal group was a manifestation of mRNA vaccine effectiveness after the first 3 weeks. Janssen vaccinees had clusters of unspecified or common vaccine reactions, gait/mobility abnormalities, and muscle weakness. The latter two were deemed to have arisen from confounding related to practices at one site. CONCLUSIONS: We detected post-vaccination clusters of unspecified adverse effects, common vaccine reactions, and, for the mRNA vaccines, chest pain and palpitations, as well as myocarditis/pericarditis after Pfizer-BioNTech Dose 2. Unique advantages of this data mining are its untargeted nature and its inherent adjustment for the multiplicity of diagnoses and risk intervals scanned.

Sequential Data-Mining for Adverse Events After Recombinant Herpes Zoster Vaccination Using the Tree-Based Scan Statistic.

Tree-based scan statistics have been successfully used to study the safety of several vaccines without prespecifying health outcomes of concern. In this study, the binomial tree-based scan statistic was applied sequentially to detect adverse events in days 1-28 compared with days 29-56 after recombinant herpes zoster (RZV) vaccination, with 5 looks at the data and formal adjustment for the repeated analyses over time. IBM MarketScan data on commercially insured persons ≥50 years of age receiving RZV during January 1, 2018, to May 5, 2020, were used. With 999,876 doses of RZV included, statistically significant signals were detected only for unspecified adverse effects/complications following immunization, with attributable risks as low as 2 excess cases per 100,000 vaccinations. Ninety percent of cases in the signals occurred in the week after vaccination and, based on previous studies, likely represent nonserious events like fever, fatigue, and headache. Strengths of our study include its untargeted nature, self-controlled design, and formal adjustment for repeated testing. Although the method requires prespecification of the risk window of interest and may miss some true signals detectable using the tree-temporal variant of the method, it allows for early detection of potential safety problems through early initiation of ongoing monitoring.

Monitoring Drug Safety in Pregnancy with Scan Statistics: A Comparison of Two Study Designs.

BACKGROUND: Traditional surveillance of adverse infant outcomes following maternal medication exposures relies on pregnancy exposure registries, which are often underpowered. We characterize the statistical power of TreeScan, a data mining tool, to identify potential signals in the setting of perinatal medication exposures and infant outcomes. METHODS: We used empirical data to inform background incidence of major congenital malformations and other birth conditions. Statistical power was calculated using two probability models compatible with TreeScan, Bernoulli and Poisson, while varying the sample size, magnitude of the risk increase, and incidence of a specified outcome. We also simulated larger referent to exposure matching ratios when using the Bernoulli model in the setting of fixed N:1 propensity score matching. Finally, we assessed the impact of outcome misclassification on power. RESULTS: The Poisson model demonstrated greater power to detect signals than the Bernoulli model across all scenarios and suggested a sample size of 4,000 exposed pregnancies is needed to detect a twofold increase in risk of a common outcome (approximately 8 per 1,000) with 85% power. Increasing the fixed matching ratio with the Bernoulli model did not reliably increase power. An outcome definition with high sensitivity is expected to have somewhat greater power to detect signals than an outcome definition with high positive predictive value. CONCLUSIONS: Use of the Poisson model with an outcome definition that prioritizes sensitivity may be optimal for signal detection. TreeScan is a viable method for surveillance of adverse infant outcomes following maternal medication use.

Assessing medical product safety during pregnancy using parameterizable tools in the sentinel distributed database.

PURPOSE: The US Food and Drug Administration established the Sentinel System to monitor the safety of medical products. A component of this system includes parameterizable analytic tools to identify mother-infant pairs and evaluate infant outcomes to enable the routine monitoring of the utilization and safety of drugs used in pregnancy. We assessed the feasibility of using the data and tools in the Sentinel System by assessing a known association between topiramate use during pregnancy and oral clefts in the infant. METHODS: We identified mother-infant pairs using the mother-infant linkage table from six data partners contributing to the Sentinel Distributed Database from January 1, 2000, to September 30, 2015. We compared mother-infant pairs with first-trimester exposure to topiramate to mother-infant pairs that were topiramate-unexposed or lamotrigine-exposed and used a validated algorithm to identify oral clefts in the infant. We estimated adjusted risk ratios through propensity score stratification. RESULTS: There were 2007 topiramate-exposed and 1 066 086 unexposed mother-infant pairs in the main comparison. In the active-comparator analysis, there were 1996 topiramate-exposed and 2859 lamotrigine-exposed mother-infant pairs. After propensity score stratification, the odds ratio for oral clefts was 2.92 (95% CI: 1.43, 5.93) comparing the topiramate-exposed to unexposed groups and 2.72 (95% CI: 0.75, 9.93) comparing the topiramate-exposed to lamotrigine-exposed groups. CONCLUSIONS: We found an increased risk of oral clefts after topiramate exposure in the first trimester in the Sentinel database. These results are similar to prior published observational study results and demonstrate the ability of Sentinel's data and analytic tools to assess medical product safety in cohorts of mother-infant pairs in a timely manner.

Tree-based data mining for safety assessment of first COVID-19 booster doses in the Vaccine Safety Datalink.

BACKGROUND: The Centers for Disease Control and Prevention's Vaccine Safety Datalink (VSD) has been performing safety surveillance for COVID-19 vaccines since their earliest authorization in the United States. Complementing its real-time surveillance for pre-specified health outcomes using pre-specified risk intervals, the VSD conducts tree-based data-mining to look for clustering of a broad range of health outcomes after COVID-19 vaccination. This study's objective was to use this untargeted, hypothesis-generating approach to assess the safety of first booster doses of Pfizer-BioNTech (BNT162b2), Moderna (mRNA-1273), and Janssen (Ad26.COV2.S) COVID-19 vaccines. METHODS: VSD enrollees receiving a first booster of COVID-19 vaccine through April 2, 2022 were followed for 56 days. Incident diagnoses in inpatient or emergency department settings were analyzed for clustering within both the hierarchical ICD-10-CM code structure and the follow-up period. The self-controlled tree-temporal scan statistic was used, conditioning on the total number of cases for each diagnosis. P-values were estimated by Monte Carlo simulation; p = 0.01 was pre-specified as the cut-off for statistical significance of clusters. RESULTS: More than 2.4 and 1.8 million subjects received Pfizer-BioNTech and Moderna boosters after an mRNA primary series, respectively. Clusters of urticaria/allergy/rash were found during Days 10-15 after the Moderna booster (p = 0.0001). Other outcomes that clustered after mRNA boosters, mostly with p = 0.0001, included unspecified adverse effects, common vaccine-associated reactions like fever and myalgia, and COVID-19. COVID-19 clusters were in Days 1-10 after booster receipt, before boosters would have become effective. There were no noteworthy clusters after boosters following primary Janssen vaccination. CONCLUSIONS: In this untargeted data-mining study of COVID-19 booster vaccination, a cluster of delayed-onset urticaria/allergy/rash was detected after the Moderna booster, as has been reported after Moderna vaccination previously. Other clusters after mRNA boosters were of unspecified or common adverse effects and COVID-19, the latter evidently reflecting immunity to COVID-19 after 10 days.

Novel methods for pregnancy drug safety surveillance in the FDA Sentinel System.

PURPOSE: It is a priority of the US Food and Drug Administration (FDA) to monitor the safety of medications used during pregnancy. Pregnancy exposure registries and cohort studies utilizing electronic health record data are primary sources of information but are limited by small sample sizes and limited outcome assessment. TreeScan™, a statistical data mining tool, can be applied within the FDA Sentinel System to simultaneously identify multiple potential adverse neonatal and infant outcomes after maternal medication exposure. METHODS: We implemented TreeScan using the Sentinel analytic tools in a cohort of linked live birth deliveries and infants nested in the IBM MarketScan® Research Database. As a case study, we compared first trimester fluoroquinolone use and cephalosporin use. We used the Bernoulli and Poisson TreeScan statistics with compatible propensity score-based study designs for confounding control (matching and stratification) and used multiple propensity score models with various strategies for confounding control to inform best practices. We developed a hierarchical outcome tree including major congenital malformations and outcomes of gestational length and birth weight. RESULTS: A total of 1791 fluoroquinolone-exposed and 8739 cephalosporin-exposed mother-infant pairs were eligible for analysis. Both TreeScan analysis methods resulted in single alerts that were deemed to be due to uncontrolled confounding or otherwise not warranting follow-up. CONCLUSIONS: In this implementation of TreeScan using Sentinel analytic tools, we did not observe any new safety signals for fluoroquinolone use in the first trimester. TreeScan, with tailored or high-dimensional propensity scores for confounding control, is a valuable tool in addition to current safety surveillance methods for medications used during pregnancy.

The US Food and Drug Administration Sentinel System: a national resource for a learning health system.

The US Food and Drug Administration (FDA) created the Sentinel System in response to a requirement in the FDA Amendments Act of 2007 that the agency establish a system for monitoring risks associated with drug and biologic products using data from disparate sources. The Sentinel System has completed hundreds of analyses, including many that have directly informed regulatory decisions. The Sentinel System also was designed to support a national infrastructure for a learning health system. Sentinel governance and guiding principles were designed to facilitate Sentinel's role as a national resource. The Sentinel System infrastructure now supports multiple non-FDA projects for stakeholders ranging from regulated industry to other federal agencies, international regulators, and academics. The Sentinel System is a working example of a learning health system that is expanding with the potential to create a global learning health system that can support medical product safety assessments and other research.

Invited Commentary: Go BIG and Go Global-Executing Large-Scale, Multisite Pharmacoepidemiologic Studies Using Real-World Data.

At the time medical products are approved, we rarely know enough about their comparative safety and effectiveness vis-à-vis alternative therapies to advise patients and providers. Postmarket generation of evidence on rare adverse events following medical product exposure increasingly requires analysis of millions of longitudinal patient records that can provide complete capture of data on patient experiences. In the accompanying article by Pradhan et al. (Am J Epidemiology. 2022;191(8):1352-1367), the authors demonstrate how observational database studies are often the most practical approach, provided these databases are carefully chosen to be "fit for purpose." Distributed data networks with common data models have proliferated in the last 2 decades in pharmacoepidemiology, allowing efficient capture of patient data in a standardized and structured format across disparate real-world data sources. Use of common data models facilitates transparency by allowing standardized programming approaches that can be easily reproduced. The distributed data network architecture, combined with a common data approach, supports not only multisite observational studies but also pragmatic clinical trials. It also helps bridge international boundaries and further increases the sample size and diversity of study populations.

A Broad Safety Assessment of the Recombinant Herpes Zoster Vaccine.

The recombinant herpes zoster vaccine (RZV), approved as a 2-dose series in the United States in October 2017, has proven highly effective and generally safe. However, a small risk of Guillain-Barré syndrome after vaccination was identified after approval, and questions remain about other possible adverse events. This data-mining study assessed RZV safety in the United States using the self-controlled tree-temporal scan statistic, scanning data on thousands of diagnoses recorded during follow-up to detect any statistically unusual temporal clustering of cases within a large hierarchy of diagnoses. IBM MarketScan data on commercially insured persons at least 50 years of age receiving RZV between January 1, 2018, and May 5, 2020, were used, including 56 days of follow-up; 1,014,329 doses were included. Statistically significant clustering was found within a few days of vaccination for unspecified adverse effects, complications, or reactions to immunization or other medical substances/care; fever; unspecified allergy; syncope/collapse; cellulitis; myalgia; and dizziness/giddiness. These findings are consistent with the known safety profile of this and other injected vaccines. No cluster of Guillain-Barré syndrome was detected, possibly due to insufficient sample size. This signal-detection method has now been applied to 5 vaccines, with consistently plausible results, and seems a promising addition to vaccine-safety evaluation methods.

Utility of fertility procedures and prenatal tests to estimate gestational age for live-births and stillbirths in electronic health plan databases.

PURPOSE: Current algorithms to evaluate gestational age (GA) during pregnancy rely on hospital coding at delivery and are not applicable to non-live births. We developed an algorithm using fertility procedures and fertility tests, without relying on delivery coding, to develop a novel GA algorithm in live-births and stillbirths. METHODS: Three pregnancy cohorts were identified from 16 health-plans in the Sentinel System: 1) hospital admissions for live-birth, 2) hospital admissions for stillbirth, and 3) medical chart-confirmed stillbirths. Fertility procedures and prenatal tests, recommended within specific GA windows were evaluated for inclusion in our GA algorithm. Our GA algorithm was developed against a validated delivery-based GA algorithm in live-births, implemented within a sample of chart-confirmed stillbirths, and compared to national estimates of GA at stillbirth. RESULTS: Our algorithm, including fertility procedures and 11 prenatal tests, assigned a GA at delivery to 97.9% of live-births and 92.6% of stillbirths. For live-births (n = 4 701 207), it estimated GA within 2 weeks of a reference delivery-based GA algorithm in 82.5% of pregnancies, with a mean difference of 3.7 days. In chart-confirmed stillbirths (n = 49), it estimated GA within 2 weeks of the clinically recorded GA at delivery for 80% of pregnancies, with a mean difference of 11.1 days. Implementation of the algorithm in a cohort of stillbirths (n = 40 484) had an increased percentage of deliveries after 36 weeks compared to national estimates. CONCLUSIONS: In a population of primarily commercially-insured pregnant women, fertility procedures and prenatal tests can estimate GA with sufficient sensitivity and accuracy for utility in pregnancy studies.

Validity of ICD-10-CM diagnoses to identify hospitalizations for serious infections among patients treated with biologic therapies.

PURPOSE: Identifying hospitalizations for serious infections among patients dispensed biologic therapies within healthcare databases is important for post-marketing surveillance of these drugs. We determined the positive predictive value (PPV) of an ICD-10-CM-based diagnostic coding algorithm to identify hospitalization for serious infection among patients dispensed biologic therapy within the FDA's Sentinel Distributed Database. METHODS: We identified health plan members who met the following algorithm criteria: (1) hospital ICD-10-CM discharge diagnosis of serious infection between July 1, 2016 and August 31, 2018; (2) either outpatient/emergency department infection diagnosis or outpatient antimicrobial treatment within 7 days prior to hospitalization; (3) inflammatory bowel disease, psoriasis, or rheumatological diagnosis within 1 year prior to hospitalization, and (4) were dispensed outpatient biologic therapy within 90 days prior to admission. Medical records were reviewed by infectious disease clinicians to adjudicate hospitalizations for serious infection. The PPV (95% confidence interval [CI]) for confirmed events was determined after further weighting by the prevalence of the type of serious infection in the database. RESULTS: Among 223 selected health plan members who met the algorithm, 209 (93.7% [95% CI, 90.1%-96.9%]) were confirmed to have a hospitalization for serious infection. After weighting by the prevalence of the type of serious infection, the PPV of the ICD-10-CM algorithm identifying a hospitalization for serious infection was 80.2% (95% CI, 75.3%-84.7%). CONCLUSIONS: The ICD-10-CM-based algorithm for hospitalization for serious infection among patients dispensed biologic therapies within the Sentinel Distributed Database had 80% PPV for confirmed events and could be considered for use within pharmacoepidemiologic studies.

Validation of an electronic algorithm for Hodgkin and non-Hodgkin lymphoma in ICD-10-CM.

PURPOSE: Lymphoma is a health outcome of interest for drug safety studies. Studies using administrative claims data require the accurate identification of lymphoma cases. We developed and validated an International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM)-based algorithm to identify lymphoma in healthcare claims data. METHODS: We developed a three-component algorithm to identify patients aged ≥15 years who were newly diagnosed with Hodgkin (HL) or non-Hodgkin (NHL) lymphoma from January 2016 through July 2018 among members of four Data Partners within the FDA's Sentinel System. The algorithm identified potential cases as patients with ≥2 ICD-10-CM lymphoma diagnosis codes on different dates within 183 days; ≥1 procedure code for a diagnostic procedure (e.g., biopsy, flow cytometry) and ≥1 procedure code for a relevant imaging study within 90 days of the first lymphoma diagnosis code. Cases identified by the algorithm were adjudicated via chart review and a positive predictive value (PPV) was calculated. RESULTS: We identified 8723 potential lymphoma cases via the algorithm and randomly sampled 213 for validation. We retrieved 138 charts (65%) and adjudicated 134 (63%). The overall PPV was 77% (95% confidence interval: 69%-84%). Most cases also had subtype information available, with 88% of cases identified as NHL and 11% as HL. CONCLUSIONS: Seventy-seven percent of lymphoma cases identified by an algorithm based on ICD-10-CM diagnosis and procedure codes and applied to claims data were true cases. This novel algorithm represents an efficient, cost-effective way to target an important health outcome of interest for large-scale drug safety and public health surveillance studies.

A COVID-19-ready public health surveillance system: The Food and Drug Administration's Sentinel System.

The US Food and Drug Administration's Sentinel System was established in 2009 to use routinely collected electronic health data for improving the national capability to assess post-market medical product safety. Over more than a decade, Sentinel has become an integral part of FDA's surveillance capabilities and has been used to conduct analyses that have contributed to regulatory decisions. FDA's role in the COVID-19 pandemic response has necessitated an expansion and enhancement of Sentinel. Here we describe how the Sentinel System has supported FDA's response to the COVID-19 pandemic. We highlight new capabilities developed, key data generated to date, and lessons learned, particularly with respect to working with inpatient electronic health record data. Early in the pandemic, Sentinel developed a multi-pronged approach to support FDA's anticipated data and analytic needs. It incorporated new data sources, created a rapidly refreshed database, developed protocols to assess the natural history of COVID-19, validated a diagnosis-code based algorithm for identifying patients with COVID-19 in administrative claims data, and coordinated with other national and international initiatives. Sentinel is poised to answer important questions about the natural history of COVID-19 and is positioned to use this information to study the use, safety, and potentially the effectiveness of medical products used for COVID-19 prevention and treatment.

Health outcomes coding trends in the US Food and Drug Administration's Sentinel System during transition to International Classification of Diseases-10 coding system: A brief review.

BACKGROUND AND PURPOSE: The transition from International Classification of Diseases, 9th revision, clinical modification (ICD-9-CM) to ICD-10-CM poses a challenge to epidemiologic studies that use diagnostic codes to identify health outcomes and covariates. We evaluated coding trends in health outcomes in the US Food and Drug Administration's Sentinel System during the transition. METHODS: We reviewed all health outcomes coding trends reports on the Sentinel website through November 30, 2019 and analyzed trends in incidence and prevalence across the ICD-9-CM and ICD-10-CM eras by visual inspection. RESULTS: We identified 78 unique health outcomes (22 acute, 32 chronic, and 24 acute or chronic) and 140 time-series graphs of incidence and prevalence. The reports also included code lists and code mapping methods used. Of the 140 graphs reviewed, 81 (57.9%) showed consistent trends across the ICD-9-CM and ICD-10-CM eras, while 51 (36.4%) and 8 (5.7%) graphs showed inconsistent and uncertain trends, respectively. Chronic HOIs and acute/chronic HOIs had higher proportions of consistent trends in prevalence definitions (83.9% and 78.3%, respectively) than acute HOIs (28.6%). For incidence, 55.6% of acute HOIs showed consistent trends, while 41.2% of chronic HOIs and 39.3% of acute/chronic HOIs showed consistency. CONCLUSIONS: Researchers using ICD-10-CM algorithms obtained by standardized mappings from ICD-9-CM algorithms should assess the mapping performance before use. The Sentinel reports provide a valuable resource for researchers who need to develop and assess mapping strategies. The reports could benefit from additional information about the algorithm selection process and additional details on monthly incidence and prevalence rates. KEY POINTS: We reviewed health outcomes coding trends reports on the US FDA Sentinel website through November 30, 2019 and analyzed trends in incidence and prevalence across the International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) and ICD-10-CM eras by code mapping method and the type of health outcomes of interest (acute, chronic, acute or chronic). More than a third of the 140 time-series graphs of incidence and prevalence of health outcomes showed inconsistent or uncertain trends. Consistency in trends varied by code mapping method, type of health outcomes of interest, and whether the measurement was incidence or prevalence. Studies using ICD-9-CM-based algorithms mapped to ICD-10-CM codes need to assess the performance of the mappings and conduct manual refinement of the algorithms as needed before using them.

A General Propensity Score for Signal Identification Using Tree-Based Scan Statistics.

The tree-based scan statistic (TreeScan; Martin Kulldorff, Harvard Medical School, Boston, Massachusetts) is a data-mining method that adjusts for multiple testing of correlated hypotheses when screening thousands of potential adverse events for signal identification. Simulation has demonstrated the promise of TreeScan with a propensity score (PS)-matched cohort design. However, it is unclear which variables to include in a PS for applied signal identification studies to simultaneously adjust for confounding across potential outcomes. We selected 4 pairs of medications with well-understood safety profiles. For each pair, we evaluated 5 candidate PSs with different combinations of 1) predefined general covariates (comorbidity, frailty, utilization), 2) empirically selected (data-driven) covariates, and 3) covariates tailored to the drug pair. For each pair, statistical alerting patterns were similar with alternative PSs (≤11 alerts in 7,996 outcomes scanned). Inclusion of covariates tailored to exposure did not appreciably affect screening results. Inclusion of empirically selected covariates can provide better proxy coverage for confounders but can also decrease statistical power. Unlike tailored covariates, empirical and predefined general covariates can be applied "out of the box" for signal identification. The choice of PS depends on the level of concern about residual confounding versus loss of power. Potential signals should be followed by pharmacoepidemiologic assessment where confounding control is tailored to the specific outcome(s) under investigation.

A Broad Safety Assessment of the 9-Valent Human Papillomavirus Vaccine.

Parents indicate that safety is their top concern about human papillomavirus (HPV) vaccination. A data-mining method not requiring prespecification of health outcome(s) or postexposure period(s) of potentially increased risk can be used to identify possible associations between an exposure and any of thousands of medically attended health outcomes; this method was applied to data on the 9-valent HPV vaccine (HPV9) to detect potential safety problems. Data on 9- to 26-year-olds who had received HPV9 vaccine between November 4, 2016, and August 5, 2018, inclusive, were extracted from the MarketScan database and analyzed for statistically significant clustering of incident diagnoses within the hierarchy of diagnoses coded using the International Classification of Diseases and temporally within the 1 year after vaccination, using the self-controlled tree-temporal scan statistic and TreeScan software. Only 56 days of postvaccination enrollment was required; subsequent follow-up was censored at disenrollment. Multiple testing was adjusted for. The analysis included 493,089 doses of HPV9. Almost all signals resulted from temporal confounding, not unexpected with a 1-year follow-up period. The only plausible signals were for nonspecific adverse events (e.g., injection-site reactions, headache) on days 1-2 after vaccination, with attributable risks as low as 1 per 100,000 vaccinees. Considering the broad scope of the evaluation and the high statistical power, the findings of no specific serious adverse events should provide reassurance about this vaccine's safety.